The Mating Rituals of Deep Neural Networks: Learning Compact Feature Representations through Sexual Evolutionary Synthesis

摘要

Evolutionary deep intelligence was recently proposed as a method forachieving highly efficient deep neural network architectures over successivegenerations. Drawing inspiration from nature, we propose the incorporation ofsexual evolutionary synthesis. Rather than the current asexual synthesis ofnetworks, we aim to produce more compact feature representations bysynthesizing more diverse and generalizable offspring networks in subsequentgenerations via the combination of two parent networks. Experimental resultswere obtained using the MNIST and CIFAR-10 datasets, and showed improvedarchitectural efficiency and comparable testing accuracy relative to thebaseline asexual evolutionary neural networks. In particular, the networksynthesized via sexual evolutionary synthesis for MNIST had approximatelydouble the architectural efficiency (cluster efficiency of 34.29X and synapticefficiency of 258.37X) in comparison to the network synthesized via asexualevolutionary synthesis, with both networks achieving a testing accuracy of~97%.